The present disclosure relates to the field of computer graphics processing. Particularly disclosed herein are techniques for increasing the computational efficiency of Bayer reconstruction, or more generally de-mosaicing, of an image.
As is generally known to those skilled in the art, the image sensors employed in most color digital cameras and/or video cameras are not truly color sensors. The CCD and CMOS image sensors used only detect brightness, i.e., the amount of light that strikes a particular pixel. For many years, it has been common practice to employ various filter arrangements in connection with the sensor such that each pixel receives light of a particular color. One such arrangement is known as a Bayer filter or color filter array (CFA), and is described generally in U.S. Pat. No. 3,971,065 to Eastman Kodak naming as inventor Bryce Bayer, which is hereby incorporated by reference in its entirety.
A schematic diagram of an image 100 produced with a Bayer filter is illustrated in FIG. 1. The image comprises a plurality of pixels 101. Each pixel is disposed behind a filter responsive to one of three or more components of colored light, e.g., red, green, and blue. Because human vision is more responsive to the green wavelengths of light, it is desirable to have a higher number of green samples, as compared to the red and blue wavelengths where human vision is less sensitive. Thus, a first row 102 of the sensor has alternating green and red filtered pixels, denoted “G” and “R”. Similarly, a second row 103 of the sensor has alternating blue and green filtered pixels, denoted “B” and “G”. The rows continue in this alternating pattern to form the entire image. As taught by Bayer, this arrangement is beneficial in that: (1) it provides the most samples (half of the array) to the more important green wavelengths and (2) it provides sampling of each color that is of roughly corresponding spatial distribution.
However, there are some problems associated with this arrangement. Because the samples of each color are not coincident, the perceived quality of the image is somewhat degraded. Additionally, the true resolution of the image is somewhat less than the actual number of pixels employed. For many applications, it is desired to reconstruct the image so as to provide a red, green, and blue color sample at each pixel location. This process is known as Bayer reconstruction.
Bayer reconstruction generally involves computing the two missing color elements of each sample by estimating from adjacent pixels, for example, as a weighted average. For instance, the green value of pixel 104, which is sampled as a red pixel, can be determined as a function of the three adjoining green pixels. Similarly, the blue value of pixel 104 can be determined as a function of the two adjacent blue pixels. Additionally, it is not necessary to reference only pixels of the same color, as some algorithms reference adjoining pixels of all colors. A variety of these techniques are known in the art and are thus not reproduced in detail here.
One common problem with all of these techniques is computational intensity. Because two missing color values must be computed for each pixel, and these missing values are computed as a function of several surrounding pixels, the number of computations required increases dramatically with image size. Typical CPU based Bayer reconstruction algorithms used in professional photo editing applications may require as much as 30 seconds to process a high resolution image. Therefore what is needed in the art is a technique to increase the computational efficiency of Bayer reconstruction.